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Abstract:  Acceptance  testing  is  considered  a  final  stage  of  validation,  and  performing 
physical  acceptance  tests  of  an  unmanned  ground  vehicle  (UGV)  can  be  expensive  and 
time-consuming.  Therefore,  this  paper  discusses  simulation-based  acceptance  testing  and 
failure  analysis  for  UGVs,  which  can  be  potentially  used  as  a  complementary  tool  in  the 
design  of  efficient  physical  acceptance  tests  for  reducing  the  testing  time  and  cost  for 
UGV  products.  In  this  paper,  both  dynamic  and  static  simulation  models  are  developed. 
A  systematic  statistical  testing  approach  is  presented  to  quantitatively  assess  when  a 
simple  static  simulation  model  can  be  used  to  approximate  a  complex  dynamic 
simulation.  The  results  from  the  simulations  show  that  a  static  simulation  model  can  be 
appropriately  used  for  determining  the  required  joint  motor  torques  of  UGVs  under  slow 
operation  speeds.  One  additional  benefit  of  using  a  static  simulation  model  is  that  it  can 
explicitly  identify  the  safe  operation  boundaries  for  preventing  UGVs  from  joint  torque 
saturation  and  rollover  failures,  thus  improving  the  UGVs’  operational  reliability.  On  the 
other  hand,  the  paper  also  shows  that  a  dynamic  simulation  model  is  indeed  needed  to 
determine  the  maximum  allowable  moving  speeds  for  UGVs  to  be  safely  operated  on 
roads  with  various  levels  of  roughness  and  bumpiness. 
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1  INTRODUCTION 

In  2004,  the  US  army  was  using  about  160  robots  in  Iraq  and  Afghanistan  (Purdy, 
2007).  This  number  grew  to  approximately  4,000  in  2007  and  continued  to  climb  to 
about  8,000  in  2010.  With  the  rapid  increase  of  unmanned  ground  vehicle  (UGV)  usage 
in  military  operations,  one  primary  concern  of  robotics  researchers  and  users  is  UGV 
reliability.  Studies  of  mobile  robots  used  in  Urban  Search  and  Rescue  (US&R)  and 
Military  Operations  in  Urban  Terrain  (MOUT)  have  shown  a  mean  time  between  failures 
(MTBF)  in  the  field  of  6  to  20  hours,  well  below  the  desired  96  hours  as  established  by 
Test  and  Evaluation  Coordination  Office  (TECO),  part  of  the  Maneuver  Support  Center 
at  Ft.  Leonard  Wood  (Kramer  &  Murphy,  2006).  Some  of  the  failures  are  due  to 
manufacturing  defects  or  subtle  interactions  between  components,  and  these  failures 
could  be  detected  and  prevented  prior  to  the  field  deployment.  However,  other  failures 
are  due  to  uncertain  operating  environments,  misuse  by  operators,  and  insufficient 
understanding  of  failure  modes.  Therefore,  it  is  important  to  develop  an  acceptance  test 
to  provide  better  understanding  of  the  failure  modes  and  to  ensure  that  such  systems  meet 
their  reliability  goals.  Although  such  testing  methods  are  widely  used  in  various 
engineering  applications,  there  is  still  no  general  guidance  for  UGV  acceptance  tests  in 
terms  of  system  reliability.  Thus,  the  purpose  of  this  paper  is  to  suggest  research  ideas 
that  might  provide  a  basis  for  the  development  of  an  acceptance  test  for  small  UGVs  or 
mobile  robots. 

Some  of  the  earliest  work  on  UGV  reliability  and  acceptance  testing  was  done  by 
Murphy,  et  al.  (Carlson,  Murphy  &  Nelson,  2004;  Carlson  &  Murphy  ,  2003;  2005; 
Carlson,  2004;  Kramer  &  Murphy,  2006).  Murphy  discussed  a  methodology  similar  to  a 
final  factory  acceptance  test,  i.e.,  a  test  usually  performed  by  a  manufacturer  prior  to 
shipping.  In  contrast,  this  paper  discusses  a  methodology  for  simulation-based 
acceptance  testing  in  which  simulations  are  used  to  identify  the  safe  operation  boundaries 
or  the  worst-case  scenarios  in  UGV  operations. 

The  proposed  simulation  approach  is  considered  as  a  complementary  tool  for 
improving  the  physical  acceptance  tests,  which  has  the  following  advantages  by 
overcoming  some  of  the  limitations  of  the  physical  tests: 

1)  The  simulation  approach  is  time  and  cost  effective  since  it  does  not  require 
expensive  physical  test  facilities,  expert  operators  to  work  with  the  UGV  during  the 
test,  or  various  test  apparatus. 

2)  Simulation  testing  can  be  conducted  on  the  full  operational  range  of  the  UGV 
beyond  the  limitations  that  the  test  facilities  and  apparatus  may  offer. 

3)  The  simulation  results  can  be  used  to  develop  the  boundaries  of  safe  operation  for 
UGVs  for  various  scenarios.  These  boundaries  can  be  potentially  used  for  (i) 
improving  the  UGV  design  to  prevent  UGVs  from  functional  failures  due  to  design 
deficiencies;  (ii)  developing  an  intelligent  human-UGV  interface  to  avoid  human 
operational  failures;  (iii)  providing  a  systematic  guideline  in  the  design  of  the 
essential  physical  testing  scenarios  for  reducing  the  physical  test  time  and  cost. 

4)  It  can  help  identify  an  optimal  design  and/or  operational  scheme,  which  exists 
beyond  the  currently  available  UGV  products. 

In  this  paper,  two  sets  of  commonly  encountered  UGV  failure  scenarios  are 
investigated:  the  first  set  studies  joint  torque  saturation  and  rollover  when  the  UGV 
attempts  to  lift  various  loads;  and  the  second  set  focuses  on  the  suspension  system  and 
flip  over  failures  when  the  vehicle  is  operated  on  bumpy  terrains  with  mild  roughness.  To 
determine  the  UGV  safe  operating  range,  we  examined  both  dynamic  and  static 
simulation  models  for  a  wheeled  ground  vehicle  whose  size  and  weight  is  within  the 
typical  range  of  a  small  mobile  robot. 


3 


UNCLASSIFIED 


The  paper  is  organized  as  follows.  Section  2  discusses  related  works  in  UGV 
reliability  and  acceptance  testing.  Section  3  presents  our  simulation-based  acceptance 
testing  methodology.  Section  4  studies  two  failure  modes  associated  with  UGV 
operations:  the  joint  saturation  of  the  robot  actuators  and  the  rollover  failure.  To  perform 
the  study,  we  conduct  both  static  and  dynamic  simulations  and  compare  their 
performances  using  statistical  hypothesis  testing.  Section  5  presents  a  similar  analysis 
concerning  two  other  failure  modes:  excessive  mechanical  shocks  and  vehicle  flip  over. 
Finally,  the  concluding  remarks  are  outlined  in  the  final  section. 

2  ACCEPTANCE  TESTING  PAST  AND  PRESENT 

This  paper  is  intended  to  help  users  of  UGVs  gain  a  better  understanding  of 
performance  and  reliability  of  a  vehicle  in  various  operational  environments.  Therefore, 
it  is  important  to  understand  failure  modes  associated  with  current  UGV  systems.  In 
(Carlson,  Murphy,  &  Nelson,  2004),  a  novel  taxonomy  of  UGV  failures  is  introduced 
which  classifies  failures  into  two  major  categories:  physical  and  human  operational 
failures.  The  physical  failures  are  further  studied  based  on  the  most  common  subsystems 
used  in  UGV  platforms:  effector,  sensor,  control  system,  power,  and  communications. 
Human  operational  failures  are  subdivided  into  misdesign  specifications  and  human- 
robot  interaction. 

The  previous  work  by  the  Center  for  Robot  Assisted  Search  and  Rescue  (CRASAR) 
includes  13  studies  and  15  different  models  of  field  robots  in  US&R  or  military 
operations  (Carlson  &  Murphy,  2003;  Carlson,  2004).  This  study  showed  an  overall 
MTBF  of  8  hours  and  an  availability  of  less  than  50%.  The  effectors  were  the  most 
common  type  of  failures,  39%  of  overall  failures,  and  the  control  system  was  the  next 
with  29%. 

In  order  to  ensure  that  field  robots  meet  such  performance  requirements  and 
reliability  targets,  it  is  important  to  develop  acceptance  and  performance  testing 
standards  for  UGVs.  Some  preliminary  work  on  UGV  acceptance  testing  has  been  done 
by  Murphy,  et  al.  In  (Kramer  &  Murphy,  2006),  the  role  of  endurance  testing  for  rescue 
and  safety  robots  is  discussed.  It  describes  a  methodology  for  endurance  testing 
recommended  for  a  certain  class  of  robots.  A  six-hour  endurance  test  was  developed  for  a 
commercially  available  rescue  robot.  The  test  uncovered  failures  under  certain  conditions 
and  the  sources  of  the  failures.  In  addition,  the  test  data  identified  key  design  and 
manufacturing  issues.  Endurance  and  agility  tests  were  also  conducted  at  the  U.S.  Army 
Aberdeen  Test  Center  (ATC)  on  small  size  UGVs.  The  robots  were  tested  under  a  variety 
of  operating  conditions  to  ensure  their  reliability  in  the  field  (Tricomo,  2009). 

In  terms  of  performance  standards,  the  Department  of  Homeland  Security  (DHS) 
initiated  an  effort  in  2004  to  develop  performance  standards  for  US&R  robots  (Messina 
&  Jacoff,  2006).  In  order  to  ensure  that  applicable  technologies  are  relatively  easy  to  use 
and  to  be  integrated  efficiently  into  existing  systems,  standardized  test  methods  were 
needed.  Therefore,  the  DHS  Science  and  Technology  Directorate  initiated  an  effort  in 
2004  with  NIST  to  develop  comprehensive  standards  to  support  development,  testing, 
and  certification  of  effective  robotic  technologies  for  US&R  applications.  These 
standards  address  robot  mobility,  sensing,  navigation,  and  human  system  interaction. 
Recently,  Pepper,  Balakirsky,  &  Scrapper  (2007)  used  the  standardized  physical  tests  in 
NIST  to  validate  the  UGV’s  simulation  models  under  the  given  certain  scenarios. 

There  are  major  limitations  that  almost  all  of  the  above  acceptance  and  performance 
testing  share.  Due  to  the  limited  testing  facilities  and  apparatus,  often  only  a  small  range 
of  operational  requirements  can  be  physically  tested.  Additionally,  conducting  physical 
tests  can  be  time  consuming  and  tedious  .  Moreover,  there  are  many  uncontrollable  or 
hard-to-control  factors  in  physical  tests  such  as  different  operator’s  knowledge  and 
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operating  skills,  environmental  condition  changes,  etc.,  which  may  lead  to  inevitable 
variations  in  the  physical  testing  results. 

In  contrast  to  the  physical  tests,  the  simulation-based  tests  can  be  very  time  efficient 
since  they  can  be  fully  automated  and  a  variety  of  scenarios  can  be  easily  integrated  into 
the  simulation.  Moreover,  simulation-based  evaluation  methods  have  played  an 
increasing  role  in  complementing  physical  tests  in  other  industries,  such  as  aerospace  and 
automotive  (Norris  1995;  Guonian,  et  al.  2010).  In  summary,  the  advantages  that  the 
simulation-based  acceptance  testing  offers,  makes  it  a  complementary  tool  for  physical 
acceptance  testing.  Therefore,  this  paper  investigates  simulation-based  acceptance  testing 
of  UGVs  to  provide  a  faster  and  easier  method  to  develop  performance  testing  and  to 
determine  robot  reliability. 

3  PROPOSED  METHODOLOGY 

Acceptance  tests  play  an  important  role  in  the  verification  and  demonstration  of  key 
performance  requirements  and  system  reliability  of  UGVs.  In  order  to  establish  an 
acceptance  test,  the  essential  performance  requirements  and  efficient  test  scenarios  for 
each  of  the  performance  requirements  need  to  be  determined.  These  test  scenarios 
emulate  UGV  operations  and  user  environments,  and  ensure  that  the  system  meets  the 
performance  requirements  and  reliability  goals. 

In  this  paper,  we  propose  a  methodology  for  simulation-based  acceptance  testing 
which  provides  an  efficient  complement  to  the  physical  acceptance  testing.  To  conduct 
the  study,  static  and  dynamic  simulations  for  a  few  typical  acceptance  testing  scenarios 
are  developed.  The  settings  represent  tasks  that  are  conventional  in  UGV  operations  and 
missions.  These  test  plans  include  several  UGV  failure  scenarios  consisting  of  joint 
torque  saturation,  rollover,  suspension  system  failure  and  flip  over  failures. 

Our  simulation-based  acceptance  testing  is  based  on  two  models:  dynamic  and  static 
simulation  models.  Each  simulation  model  offers  advantages  and  disadvantages,  and 
Table  1  provides  a  qualitative  summary  of  the  key  differences  between  them.  Dynamic 
simulation  can  include  a  variety  of  environmental  conditions  which  makes  its  results 
more  accurate,  whereas  static  simulation  is  more  straightforward  and  quick  to  construct. 
Using  the  static  simulation,  we  can  readily  derive  the  boundaries  of  safe  operation 
through  closed  form  static  equations,  while,  in  the  dynamic  case,  these  boundaries  are 
obtained  by  exploring  the  large  space  of  dynamic  parameters  and  environments  through 
simulations. 

Table  1:  Comparison  of  Dynamic  and  Static  Simulations 


Characteristic 

Dynamic 

Simulation 

Static 

Simulation 

Complexity 

Complex 

Straight  forward 

Time/Cost 

Time  consuming 

Quick 

Accuracy 

Accurate 

Approximate 

Failure  Boundary  Identification 

Difficult 

Quick 

The  static  simulation  is  more  time  efficient,  but  may  not  always  be  an  acceptable 
approximation  for  the  dynamic  simulation  when  the  failure  mechanism  in  a  scenario  is 
significantly  affected  by  the  dynamic  characteristics  of  the  UGV.  Therefore,  there  is  a 
need  to  develop  a  systematic  method  to  quantitatively  ensure  when  a  static  analysis  is 
satisfactory.  For  this  purpose,  dynamic  and  static  simulation  results  are  compared  by 
proposing  a  statistical  hypothesis  testing,  which  is  used  to  judge  whether  both 
simulations  will  make  a  consistent  decision  on  the  failure  state  for  a  given  scenario.  The 
purpose  of  using  a  statistical  hypothesis  test  is  to  consider  the  inherent  performance 
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variation  due  to  operational  or  environmental  uncertainties.  As  will  be  discussed  in  the 
later  sections,  the  outcome  of  the  statistical  hypothesis  test,  as  to  when  static  simulations 
can  be  used  to  approximate  more  complex  dynamic  simulations  for  a  given  scenario,  can 
be  extended  to  similar  acceptance  testing  scenarios  that  share  the  same  fundamental 
characteristics. 

The  simulation  results  can  help  further  identify  the  boundaries  of  safe  operation,  which 
can  be  used  as  a  guideline  to  design  efficient  physical  tests  scenarios  and  prevent 
expensive  damages  to  a  UGV  during  tests.  Moreover,  from  the  design  perspective,  the 
margin  between  the  operational  state  and  the  design  specification  can  indicate  the  failure 
probability  or  risk  for  conducting  a  given  mission  task.  This  will  also  provide  us  with  a 
better  understanding  of  failure  mechanism,  and  guide  us  to  improve  the  design  of  UGVs. 


Figure  1:  Proposed  methodology  for  simulation-based  acceptance  testing  and  development  of 

boundaries  of  safe  operation. 

4  STUDY  OF  TORQUE  SATURATION  AND  ROLLOVER 
FAILURES 

4.1  Scenario  Description 

In  the  first  scenario,  the  robot  lifts  and  moves  an  object  with  a  known  mass  using  its 
manipulator  arm.  The  possible  failure  modes  associated  with  this  scenario  are  the 
inability  of  the  arm  actuators  to  provide  the  necessary  torque  for  lifting  the  load,  and  also 
UGV  rollover  during  the  same  lifting  operation. 

While  it  is  desirable  that  the  arm  movements  occur  at  a  constant  angular  velocity,  in 
reality  speed  variations  occur  specially  at  low  angular  velocities  due  to  a  variety  of 
reasons  such  as  variations  in  the  power  source,  manufacturing  limitations,  interaction 
among  components  in  the  system,  etc.  Several  papers  have  discussed  the  presence  and 
methods  to  reduce  the  negative  effect  of  speed  and  torque  ripple  in  robot  actuators  by 
utilizing  effective  control  systems  (Ren,  Yu,  &  Liang,  2009;  Lam,  Panda,  &  Xu,  2000; 
Godler,  Ohnishi,  &  Yamashita,  1994).  Nonetheless,  even  with  the  use  of  feedback 
control,  speed  variations  are  not  entirely  eliminated.  Moreover,  since  this  scenario 
primarily  focuses  on  the  actuator  torque  saturation,  and  speed  and  torque  ripples 
influence  this  measure,  their  effect  is  considered  in  the  dynamic  simulation. 

Electric  motors  are  typically  used  in  robot  actuators.  The  accuracy  of  these  motors 
mainly  depends  on  the  applications.  For  the  purpose  of  this  study,  we  assume  that  the 
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actuator  is  using  an  effective  control  system  to  eliminate  the  major  harmonic  trends  in  the 
ripples.  With  the  major  trends  removed,  the  remaining  disturbances  are  assumed  to  be 
random.  Therefore,  the  speed  ripples  are  modeled  using  white  noise.  The  magnitude  of 
the  ripples  are  considered  to  be  around  1%  of  the  maximum  operating  speed  of  the  robot 
arm,  i.e.,  0.3  -  0.5  deg/sec  for  a  maximum  speed  of  about  30-50  deg/sec. 

4.2  Study  of  Torque  Saturation  Failure 
4.2.1  Static  Simulation  Framework 

Consider  a  system  consisting  of  a  wheeled  platform  and  a  two-link  planar  robot  arm 
as  shown  in  Figure  2.  It  is  desirable  to  drive  each  link  by  a  separate  joint  motor  due  to 
ease  of  position  control  from  a  control  logic  viewpoint.  The  required  joint  torque  to 
maintain  the  link  in  a  certain  position  is  merely  the  reaction  moments  at  each  joint.  A 
two-link  planar  robot  arm  is  schematically  shown  in  Figure  3.  Solving  for  the  reaction 
moments,  or  required  joint  torque,  results  in  the  following  relations 


Figure  2:  Multi-body  dynamic  simulation  model  in  the  “closed-in”  position  and  the  “manipulator 

extended”  position 


Figure  3:  Schematic  of  a  Two-link  Robot  Arm 
T\  =  (ml8~Y  +  m2gL\  +  FLL1)cos(01)  +  (m2g^y  +  FLL2)cos(0l  +  02)  (1) 

t2  =(m2S^-  +  FiL2)cOS(0i  +  &2)  (2) 

In  which, 

Tj  =  The  reaction  moment  at  joint  i 
6 1  =  The  orientation  of  the  robot  arm  i 
Fl  =  Weight  of  the  load 
mL  =  Mass  of  the  load 
=  Mass  of  the  robot  arm  i 
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Equation  (1)  defines  the  reaction  moment  at  the  first  joint,  and  Equation  (2)  defines 
the  reaction  moment  at  the  second  joint.  If  6X  is  given,  rx  and  r2  have  maximum  and 

dr  i  dry 

minimum  values  with  respect  to  02  when - ,  — —  are  equal  to  zero. 

d  d 

-^  =  -^  =  -(m2gU^  +  FLL2)sin(0\  +  ^2)  =  °  (^) 

d02  d02  2 

We  denote  the  maximum  and  minimum  torques  in  the  static  simulation  with  rsmax, 
and  r™m  respectively. 

During  the  static  simulation,  the  second  joint  angle,  02 ,  is  varied  from  0  to  271,  i.e., 

one  full  revolution,  for  each  selected  first  joint  angle.  The  joint  torques  are  then 
calculated  using  the  relations  listed  above.  As  we  can  see  from  the  equations,  the 
maximum  and  minimum  for  both  t\  and  r2  are  observed  at  the  same  orientation  of  the 
robot  arm.  By  solving  Equation  (3),  we  can  conclude  that  at  the  worst-case  orientations, 
the  second  joint  angle  is  determined  as  62=2n-  #7  if  0<  0j<tt/2  and  02=n-  07  if  n/2<  07<7t. 
Given  the  first  link  angle,  the  orientation  of  the  second  link  which  results  in  maximum 
torque  experienced  by  the  robot  actuators  are  shown  in  Table  2. 

Table  2:  The  link  orientations  that  result  in  maximum  torque  experienced  by  the 

actuators. 


By  setting  the  joint  torque  thresholds  T1  and  T2 ,  the  safe  working  range  of  the  second 


joint  can  be  determined  under  the  given  first  joint  angle  range  of  0<  61<n/2. 

Tl  ^  r2  (4) 

T\  ^(m\g^  +  m2gLx  +  FLLl)cos{0l)  +  {m2g^-  +  FLL2)cos{8l  +02)  (5) 

T\  ~{m\g^r  +  m2Sk  +FlL\) cos(6>!) 

COS(#i  +  d2 )  < - - - - -  (6) 

T2>(m2g^  +  FLL2)cOs(9l+02)  (7) 

cos(#j  +  02 )  < - -  (8) 

(m2g^  +  FLL2) 
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From  Equations  (6)  and  (8),  we  can  determine  that  the  first  joint  has  a  narrower  safe 
working  range  than  the  second  joint  under  the  same  joint  torque  threshold  value  because 
the  right  hand  side  of  equation  (6)  will  always  be  smaller  than  that  of  (8).  Similar 
analysis  and  conclusion  can  be  obtained  under  the  first  joint  angle  range  of  k/2<  0j<7t. 

The  static  simulation  model  described  above  was  evaluated  under  varying  arm 
dimensions,  masses,  load  size  and  joint  angles.  The  model  used  in  the  static  simulation 
has  the  dimensions  Lj=0.55m,  L2=0.64m,  mi  =2. 5 kg,  m2=2.5kg,  mL=4kg ,  which  are  based 
on  the  measurements  taken  from  an  actual  UGV  manipulator.  The  same  parameters  are 
used  in  the  dynamic  simulation  which  will  be  presented  next. 

4.2.2  Dynamic  Simulation  Framework 

Dynamic  analysis  generally  provides  more  accurate  results  compared  with  the  static 
study  since  the  former  takes  into  account  the  arms  inertia,  interaction  between  various 
components,  and  other  disturbances  such  as  speed  variations.  However,  the  inclusion  of 
these  factors  also  increases  the  complexity  of  the  dynamic  model,  which  in  turn  imposes 
a  challenge  for  the  development  of  analytical  solutions.  Therefore,  in  most  situations,  a 
simulation  approach  has  to  be  used  to  perform  multi-body  dynamic  analysis  with  the 
consideration  of  these  factors  in  the  model. 

In  the  multi-body  dynamic  simulation,  all  the  components  are  modeled  in  a  CAD 
system  and  converted  into  rigid  bodies  for  use  in  MSC  ADAMS,  a  multi-body  dynamic 
simulation  software.  After  all  the  parts  are  assembled,  the  complete  model  is  exported 
into  MSC  ADAMS.  The  simulation  calculates  information  such  as  lateral  and 
longitudinal  forces,  torques,  angular  velocity  and  acceleration  at  each  joint.  The  model 
also  accounts  for  all  center  of  gravity  locations  in  each  component.  Figure  2  shows  the 
3-D  graphical  rendering  of  the  vehicle  model  in  the  “closed-in”  position  and  the 
“manipulator  extended”  position. 

During  the  dynamic  simulation,  for  each  selected  first  joint  angle,  6 ^  the  second  joint 
angle,  02,  is  varied  from  0  to  2n  radian,  i.e.,  one  full  revolution.  This  makes  the 
manipulator  move  through  a  full  range  of  motion  and  provides  data  for  all  operating 
states. 

4.2.3  Dynamic  and  Static  Simulations  Results  for  Joint  Torque  Saturation 

When  the  upper  threshold  of  the  joint  torque  for  the  link  actuators  are  known,  the 
failure  of  the  robot  manipulator  will  occur  when  the  joint  torque  exceeds  the  threshold. 
For  example,  assuming  that  the  joint  torque  threshold  is  50  Nm,  the  first  joint  of  the 
manipulator  used  in  the  simulation  will  fail  in  various  positions.  Various  manipulator 
orientations  are  evaluated  using  both  dynamic  and  static  simulations.  The  results  for 
dynamic  simulation  at  two  angular  velocities,  one  with  a  typical  operating  speed  of  10 
degrees/sec  and  the  other  with  a  much  larger  speed  of  80  degrees/sec,  along  with  the 
static  simulation  result  at  the  first  joint  angle  of  te/6,  with  a  joint  torque  threshold  of  50 
Nm  is  shown  in  Figure  4.  As  shown,  speed  and  torque  ripples  create  variations  in  the 
torque  measurements  in  the  dynamic  simulation  as  expected.  Due  to  larger  moment  arms, 
speed  variations  create  spikes  that  are  more  pronounced  near  the  torque  maxima  in  the 
dynamic  simulation.  Additionally,  the  maximum  torque  in  the  dynamic  simulation  does 
not  increase  significantly  within  typical  operating  speeds  of  the  arm.  In  fact,  using 
statistical  hypothesis  testing  we  will  show  that  a  straightforward  static  simulation  can 
provide  a  reasonable  estimation  for  failure  detection.  In  the  next  section,  we  will  evaluate 
the  validity  of  the  conclusion  under  the  selected  operating  speeds. 
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Figure  4:  Static  and  dynamic  simulation  torque  profiles  for  the  first  robot  joint  with  the  first  joint 
angle  of  tt/6  radians.  (Top)  operating  speed  of  10  deg/sec.  (Bottom)  operating  speed  of  80  deg/sec 


4.2.4  Statistical  Hypothesis  Test  for  Simulation  Comparison 

To  further  quantitatively  assess  the  differences  between  static  and  dynamic  simulation 
results  and  justify  whether  the  static  simulation  can  be  used  instead  of  dynamic 
simulation  for  predicting  failures,  we  will  use  statistical  hypothesis  tests.  In  this  scenario, 

the  failure  occurs  when  rjiax(<^)  exceeds  the  joint  torque  thresholds,  where  rjiax(<^) 
refers  to  the  maximum  torque  obtained  from  the  dynamic  simulation  when  the  arm  is 
traveling  at  co  degrees/sec.  Consequently,  the  comparison  between  static  and  dynamic 

simulation  would  be  based  on  the  accuracy  of  the  static  simulation  in  predicting  r^ax(^) 
within  typical  operating  speeds.  The  advantage  of  static  analysis  is  its  simplicity  and 
ease  of  use.  On  the  other  hand,  the  dynamic  model  provides  more  accurate  and  realistic 
results,  but  it  is  much  more  complex  and  time  consuming  to  construct.  As  a  result,  when 
comparing  the  dynamic  and  static  simulations,  we  would  tolerate  slight  differences  in 
their  results  by  imposing  a  threshold  for  maximum  allowable  deviation  of  static  results 
from  their  dynamic  counterparts.  This  threshold  is  set  based  on  the  maximum  deviation 
that  is  not  of  significance  to  the  end  user  when  studying  a  particular  failure  mode. 

Since  in  this  scenario  the  effect  of  speed  variations  are  included  as  a  part  of  the 
dynamic  simulation,  the  maximum  torque  during  a  full  revolution  of  robot  arm,  which  we 

will  refer  to  by  r™x(co) ,  is  a  random  variable  whose  variance  is  affected  by  the 
magnitude  and  frequency  of  the  random  speed  ripples.  If  the  maximum  operating  speed 
of  the  robot  arm  is  30  degrees/sec,  the  magnitude  of  the  speed  ripples  are  considered  to 
be  around  1%  of  this  maximum  operating  speed. 

Statistical  hypothesis  test  will  be  used  to  verify  whether  deviation  of  the  static 
simulation  from  dynamic  simulation  result  is  significant  or  not.  A  one-sided  hypothesis 
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test  on  the  differences  between  the  mean  of  the  maximum  torque  measurements  at  each 
angular  velocity  with  the  calculated  maximum  torque  in  static  simulation  is  constructed 
as  follows 


u  .  ,,  max 

tt  ...  max 


<  A 
>  A 


(9) 


In  which  A  is  the  maximum  allowable  difference  between  the  static  and  dynamic 

simulation  results,  and  //  max  is  the  mean  of  the  maximum  torque  measurements  in 

^  O) 

the  dynamic  simulation.  With  a  prior  belief  that  the  static  and  dynamic  simulations  are 
both  capable  of  predicting  failures  accurately,  the  rejection  of  null  hypothesis  in  (9)  is  an 
indication  that  the  static  simulation  is  no  longer  capable  of  predicting  the  failure  at  a 
given  allowable  deviation  level,  and  the  dynamic  simulation  should  be  used  instead. 

The  null  hypothesis  in  (9)  is  rejected  if: 

Td  ((d)  -  (rs  +A)^ 

- ^ - zta,n- 1 

4n 

In  which,  rjnax(^)  and  ST  are  the  sample  mean  and  standard  deviation  of  the 

maximum  torque  measurements.  zrJnax(<59)  and  ST  are  calculated  by  running  the  dynamic 
simulation  for  n  complete  revolutions,  collecting  the  maximum  torque  value  in  each 
revolution,  and  computing  the  corresponding  sample  mean  and  standard  deviation. 

Additionally,  ta  n_ \  is  the  100  (l  -  «)th  percentile  of  the  t  distribution  with  n-1  degrees  of 
freedom.  Inequality  (10)  can  be  rewritten  as: 

-max.  ,  ^  max  A  .  /ii\ 

Td  (cd)— Ts  -\~A-\-tan_i  i —  (11) 

yin 

The  right  hand  side  of  inequality  (11)  provides  a  rejection  boundary  for  the  hypothesis 
test.  If  rjnax(^)  is  larger  than  the  rejection  boundary,  the  hypothesis  test  is  rejected 
indicating  that  the  deviation  of  static  and  dynamic  simulations  is  significant  and  dynamic 
simulation  should  be  used  for  failure  analysis. 


4.2.5  Findings  from  the  Statistical  Hypothesis  Test 


The  result  from  the  static  analysis  agrees  with  the  result  from  the  dynamic  analysis 
with  a  typical  operating  speed  of  10  degrees/second.  Figure  5  depicts  rjnax(^)  at  various 
angular  velocities  along  with  the  hypothesis  test  rejection  boundary  obtained  from  (11). 
This  figure  shows  that  the  static  simulation  model  can  be  used  for  failure  analysis,  rather 
than  a  more  complex  dynamic  simulation  model,  due  to  slow  operating  speeds  of  UGV 
manipulator  arms.  However,  static  simulation  is  not  always  able  to  take  the  place  of 
dynamic  simulation  because  the  joint  torque  increases  as  the  robot  operating  speed 
increases  due  to  inertial  effects.  With  the  typical  maximum  operating  speed  of  about  30- 
50  degrees/second,  as  shown  in  Figure  5,  static  simulation  provides  reasonable  estimates 
for  angular  velocities  as  high  as  the  maximum  operating  speed.  However,  the  accuracy  of 
estimates  gradually  deteriorates  at  higher  angular  velocities.  The  hypothesis  test  result 
verifies  this  observation  by  rejecting  the  static  simulation  results  at  angular  velocities 
above  150%  of  the  maximum  operating  speed. 
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Angular  Velocity  of  UGV  Arms  as  a  percentage  of 
Maximum  Operation  Velocity  (%) 

Figure  5:  Statistical  comparison  between  dynamic  and  static  simulations,  for  the  first  joint  angle 

of  7i/6  radians  and  A  =  x  5%  ,n  =  10,  a  =  0.99 

4.2.6  Identifying  the  Test  Range  for  the  Joint  Torque  Saturation  Failures 

In  the  previous  section,  it  was  shown  that  static  simulation  would  provide  reasonable 
approximation  for  torque  saturation  analysis  within  typical  operating  speeds.  The 
developed  static  simulation  model  is  further  applied  to  identify  the  test  range  under 
various  joint  torque  threshold  values  for  the  joint  torque  saturation  failures.  Assuming  the 
threshold  values  are  3 (Wm,  5 OAfaz,  and  7(Wm,  the  failure  region  versus  the  safe  operating 
range  of  the  manipulator  is  shown  in  Figure  6.  This  plot  will  be  very  useful  for 
acceptance  testing  planning  because  it  shows  the  boundaries  of  operating  range  under 
different  joint  torque  thresholds.  For  example,  when  the  joint  torque  threshold  of  50Nm 
is  given,  the  circled  line  in  Figure  6  forms  a  failure  boundary.  These  failure  ranges, 
which  can  potentially  be  costly  during  the  actual  test,  should  be  avoided  and  excluded 
from  the  acceptance  testing.  The  acceptance  test  scenario  would  then  assess  the  UGV 
capabilities  in  the  safe  operating  regions. 


Second  Joint  Angle  (Radian) 

Figure  6:  Failure  range  and  safe  operating  range  of  the  two-link  planar  robot  arm  for  threshold 
values  (30,  50,  and  70Nm)  are  shown  in  terms  of  first  and  second  joint  angle. 

It  should  be  noted  that  although  both  static  and  dynamic  simulations  have  to  be 
conducted  for  the  purpose  of  comparison,  the  simulations  and  hypothesis  tests  do  not 
need  to  be  implemented  and  compared  again  when  there  are  minor  design  changes  such 
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as  changes  in  the  length  of  the  robot  arms.  Additionally,  for  acceptance  testing  scenarios 
that  share  similar  characteristics  both  simulations  do  not  need  to  be  repeated,  and  the 
previously  identified  simulation  scheme  can  be  used.  For  instance,  static  simulation  can 
be  used  for  studying  operations  with  UGV  arms  whose  range  of  operating  speeds  are 
similar  to  those  discussed  earlier  without  the  need  for  constructing  a  dynamic  simulation 
model.  In  addition,  even  if  both  simulation  schemes  are  needed  for  a  new  acceptance 
testing  scenario  that  does  not  share  the  characteristics  of  the  previous  simulated  tests,  the 
hypothesis  test  can  largely  reduce  the  analysis  time  by  identifying  conditions  in  which  a 
static  simulation  can  be  used.  In  other  words,  instead  of  exploring  the  large  space  of 
dynamic  parameters  and  environment  by  means  of  the  dynamic  simulation,  the 
hypothesis  test  result  may  suggest  that  analytical  static  simulations  can  be  used  to  quickly 
identify  the  boundaries  of  safe  operation. 

4.3  Study  of  Rollover  Failure 

4.3.1  Identifying  the  Test  Range  for  the  Rollover  Failures 

The  static  simulation  model  discussed  in  the  previous  section  can  be  combined  with 
rollover  failure  simulation.  First,  additional  model  parameters  such  as  platform 
dimensions  and  weight  are  defined.  Next,  equations  for  static  analysis  are  derived,  and 
this  static  simulation  model  is  implemented.  Several  initial  robot  orientations  can  be 
chosen  to  test  whether  the  system  rolls  over  while  the  robot  arm  travels  through  its  full 
range  of  motion,  and  those  robot  orientations  are  shown  in  Figure  7.  All  these 
orientations  are  evaluated,  and  the  result  for  the  right -tilt  orientation  is  shown  in  Figure  8. 
As  shown,  the  initial  robot  orientation  has  a  significant  impact  on  failure  and  its  safe 
operating  range,  and  this  result  can  provide  guidance  for  operators  to  avoid  rollover 
failures.  Additional  test  methods  can  be  based  on  these  initial  robot  orientations, 
including  a  dragging  capability  test  and  degradation  in  lifting  capability  without  flippers. 


Figure  7:  Example  initial  robot  orientations  for  test  set  up.  Test  measures  the  safe  operation 
range  of  the  tilt  angle  for  different  robot  arm  orientations. 
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Figure  8:  Failure  range  and  safe  operating  range  of  the  two-link  planar  robot  arm  for  Right-tilt 
(roll)  angle  of  30  and  40degree  are  shown  in  terms  of  first  and  second  joint  angle. 

5  STUDY  OF  THE  SUSPENSION  SYSTEM  AND  FLIP  OVER 
FAILURES 

5.1  Scenario  Description 

UGVs  are  used  in  a  variety  of  environmental  conditions.  As  a  result,  it  is  essential  to 
study  the  effect  of  diverse  operational  conditions,  such  as  road  bumpiness  and  roughness, 
on  the  mobility  capabilities  of  the  robot.  In  this  scenario,  the  UGV  moves  across  a  bumpy 
road  with  mild  roughness  at  various  velocities.  This  scenario  is  inspired  in  part  by  the 
acceptance  test  methods  for  evaluating  the  mobility  capabilities  of  emergency  response 
robots  conducted  by  NIST  (Jacoff  et  al.,  2009).  Figure  9  shows  the  UGV  on  the  road.  A 
few  interrelated  failure  modes  can  be  studied  under  this  setting  as  follows: 

1 :  When  the  forward  velocity  of  the  UGV  is  increased,  as  it  drives  on  a  rough  road, 
the  wheels  intermittently  lose  contact  with  the  road.  The  high  inertia  of  the  vehicle  will 
raise  the  wheels,  and  can  potentially  lead  to  loss  of  vehicle  contact  with  the  road. 

2:  After  the  loss  of  wheel  contact  with  the  road,  if  the  robot  velocity  is  further 
increased,  the  UGV  will  experience  excessive  mechanical  shocks.  These  shocks  are 
introduced  when  the  robot  wheels  come  into  contact  with  the  road  again  after  losing 
contact  initially.  They  can  result  in  failure  of  the  suspension  or  damage  electronic  devices 
on  the  vehicle. 

3:  The  UGV  may  also  flip  over  when  it  moves  at  high  velocities  on  a  rough  road 
surface.  The  loss  of  balance  usually  occurs  when  the  UGV  crosses  parts  of  the  road  that 
have  higher  inclination  angles. 

The  above  failure  modes  are  closely  related.  The  relationships  among  them  will  be 
further  elaborated  in  Section  5.6. 


Figure  9:  (Left)  UGV  crossing  (left  to  right)  a  bumpy  road.  (Right)  One- 
degree-of-  freedom  spring-damper  suspension  was  added  to  the  UGV 
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5.2  Static  Simulation  Framework 

To  conduct  the  static  simulation,  a  2-D  representation  of  the  UGV  is  considered.  The 
corresponding  free  body  diagram  and  the  static  equations  are  shown  below.  The 
numerical  values  of  parameters  used  in  the  equations  are  provided  in  Table  3.  For  the 
purpose  of  comparison  between  dynamic  and  static  simulations,  the  maximum 
suspension  force  on  a  given  road  will  be  calculated.  It  is  assumed  that  the  wheels  are 
locked  and  only  the  static  friction  between  the  wheels  and  the  road  prevents  the  UGV 
from  sliding. 

The  central  mass  of  the  vehicle  is  closest  to  the  rear  left  wheel.  Consequently,  the  rear 
left  suspension  experiences  the  largest  forces  due  to  the  asymmetrical  center  of  mass  of 
the  UGV.  Since  mechanical  shocks  are  associated  with  large  reaction  forces  in  the 
suspension,  special  care  should  be  directed  to  the  rear  left  wheel  for  the  failure  analysis 
of  the  suspension  system. 


Figure  10:  Free  Body  Diagram  of  the  UGV 

R  _  Mg(L2  cos(6>)  +  4  sin(ff)) 

1  ( k+L2 ) 

D  Mg(Z1cos(6»)-Z3sin(6>)) 

l\l  ~  - 

(L\  +L2) 

F\=  R\~  (/«,,. g)cos(<9) 

F2  =R2  ~(mwg)  cos(0) 

In  which, 

Rt  =  Reaction  force  of  the  wheel  i 
Ft  =  Suspension  force  associated  with  the  wheel  i 
Fri  =  Static  friction  between  the  wheels  and  the  road 
6  =  Orientation/angle  of  the  UGV  on  the  road 
mw  =Mass  of  the  wheel 
M  =  UGV  total  mass  including  wheels 

Figure  1 1  shows  that  the  orientation  of  the  UGV  on  the  road  has  a  significant  effect 
on  the  suspension  system  loads.  In  particular,  the  maximum  suspension  load  occurs  when 
the  UGV  is  positioned  on  parts  of  the  road  with  largest  angles.  In  addition,  the  road 
roughness  is  too  small  to  change  the  orientation  of  the  vehicle  and  consequently  their 
effect  on  the  static  simulation  is  negligible.  The  maximum  angle  of  the  UGV  on  the  road 
depends  on  the  size  of  the  UGV,  amplitude,  and  spatial  frequency  of  the  bumps  on  the 
road.  The  negative  angles  refer  to  down-slopes,  and  positive  angles  indicate  upward 


(12) 

(13) 

(14) 

(15) 
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slopes.  Figure  11  illustrates  that /q  increases  with  larger  up-slopes  (larger  positive 0), 
while  F2  increases  with  larger  down-slopes  (larger  negative  9 ). 


Static  Simulation  Results  For  Suspension  Force 
140 


UGV  Angle 


Figure  11:  The  static  simulation  results  for  the  rear  and  front  suspension  force. 
Table  3:  Parameters  used  in  the  static  and  dynamic  simulations 


Parameter  Name 

Value 

U 

21.2  cm 

u 

28.7  cm 

u 

23  cm 

UGV  Mass  in  Static  Analysis 

18.47  kg 

Wheel’s  Mass 

0.166  kg 

Coefficient  of  Static  Friction 

0.8 

Coefficient  of  Dynamic  Friction 

0.7 

Spring  Constant 

5000  N/m 

Damping  Coefficient 

250  N-Sec/m 

UGV  Typical  Arm  Operating  Speed 

10  degrees/second 

UGV  Maximum  Operating  Speed 

6  miles/hour 

Ge 

0.01 

Gs 

20 

Ga 

lxlO'6 

5.3  Dynamic  Simulation  Framework 

Similar  to  the  torque  saturation  scenario  in  section  4,  all  the  components  such  as  road 
profiles  were  modeled  in  a  CAD  system  and  converted  into  rigid  bodies  for  use  in  MSC 
ADAMS.  The  parameters  used  in  the  dynamic  simulation  such  as  coefficients  of  friction 
between  the  road  and  the  UGV  wheels,  range  of  operating  speeds,  and  suspension  system 
details  are  included  in  Table  3.  The  road  profile  is  assumed  to  follow  a  sine  function  with 
amplitude  of  0.075  meter.  The  simulation  considers  a  variety  of  spatial  frequencies  for 
the  road.  Additionally,  the  road  model  includes  mild  roughness.  The  roughness  of  the 
road  was  generated  using  MSC  ADAMS/CAR  road  generation  toolbox.  To  summarize, 
the  road  profile  follows  the  expression  below: 

Y  (x)  =  0.075  x  sin  (corx)  +  (p{Ge  ,Gs,Ga,x)  (16) 

In  which,  cor  is  the  spatial  frequency  of  the  road,  and  Ge,Gs,Ga  are  white  noise 
elevation,  slope  and  acceleration  parameters  respectively.  These  parameters  are  used  for 
generating  the  road  roughness  according  to  (Sayers,  1988).  Using  the  values  in  Table  3 
our  road  profile  represents  a  bumpy  Portland-cement  concrete  road  with  a  rigid  to 
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smooth-rigid  roughness  characteristic.  Additionally,  larger  cor  will  decrease  the  distance 
between  bumps  which  results  in  sharper  road  angles  and  higher  excitation  frequencies. 

In  order  to  capture  the  effect  of  mechanical  shocks  and  road  roughness,  a  one-degree- 
of-freedom  suspension  (spring-damper)  system  was  added  to  the  model.  Figure  9  depicts 
the  3-D  graphical  rendering  of  the  model.  The  properties  of  the  suspension  system  such 
as  spring  and  damping  coefficients  are  provided  in  Table  3.  These  parameters  were 
selected  to  provide  a  combination  of  low  natural  frequency,  and  large  enough  stiffness 
and  damping  to  prevent  excessive  spring  deformation.  The  spring  deformation  was 
critical  due  to  the  small  size  of  the  UGV  and  limitations  on  suspension  stroke.  In 
addition,  considering  the  road  profiles  and  typical  operating  speeds  of  the  UGV,  the 
excitation  of  the  suspension  system  induced  by  movement  of  the  UGV  on  the  road  will 
be  smaller  than  the  natural  frequency  of  the  suspension  system. 

The  road  roughness  in  the  dynamic  simulation  results  in  random  suspension  forces. 
The  effect  of  road  roughness  on  the  suspension  loads  is  depicted  in  Figure  12.  As  shown 
the  maximum  force  differs  slightly  from  one  road  cycle  to  the  other.  As  shown  in  the 
figure,  the  road  consists  of  several  bumps,  and  the  road  cycles  are  equality  spaced 
including  one  bump  each. 


Figure  12:  The  effect  of  road  roughness  on  force  profiles.  Comparison  of  rear  suspension  forces  on 

a  rough  versus  a  smooth  bumpy  road. 

5.4  Statistical  Hypothesis  Test  for  Simulation  Comparison 

Similar  to  the  torque  saturation  scenario,  the  goal  is  to  assess  the  static  simulation 
effectiveness  for  predicting  the  failure.  The  failure  mode  under  consideration  is  the 
suspension  system  breakdown,  which  is  caused  by  excessive  mechanical  shocks. 
Therefore,  the  measure  of  interest  in  the  comparisons  is  the  maximum  force  that  the 
suspension  experiences  on  the  road.  The  deviation  of  static  results  from  the  dynamic 
simulation  outcome  is  determined  via  statistical  tests.  A  one-sided  hypothesis  test  on  the 
differences  between  the  mean  of  the  maximum  suspension  forces  in  the  dynamic 
simulation  at  different  velocities  with  the  calculated  maximum  force  in  the  static 
simulation  is  constructed  as  follows: 

H{]  :  LI  r-.  max  -F™x  <  A 

(17) 

Hi  >  A 

In  which,  Fsmaxis  the  maximum  suspension  force  on  a  given  road  obtained  from  the 

static  simulation,  and  ju^ max.T^  is  the  mean  of  the  maximum  force  measurements  in  the 

Ai  v ) 

dynamic  simulation  when  the  robot  goes  through  one  complete  road  cycle  with  a  forward 
velocity  V. 

The  null  hypothesis  in  (17)  is  rejected  if: 
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i7dm>)-(CX  +  A) 

SF 

4n 


>t 


a,n-\ 


(18) 


In  which,  FdmaV)and  SF  are  the  sample  mean  and  standard  deviation  of  the 


maximum  force  measurements.  FdmaV)  and  SF  are  calculated  by  running  the  dynamic 
simulation  for  n  road  cycles,  collecting  the  maximum  force  value  in  each  cycle,  and 
computing  the  corresponding  sample  mean  and  standard  deviation.  Inequality  (18)  can  be 
rewritten  as: 

Frv)>rx+A+v_,^  (i9) 

yjn 

The  right  hand  side  of  the  inequality  (19)  provides  a  rejection  boundary  for  the 
hypothesis. 


5.5  Findings  from  the  Statistical  Hypothesis  Test 

Static  analysis  fails  to  predict  failure  within  typical  operating  speeds.  Figure  13 
depicts  Fdmax(T)  along  with  the  hypothesis  test  rejection  boundary  obtained  from  (19). 

The  value  of  A  is  assumed  to  be  5%  of  the  Fsmaxin  Figure  13.  The  statistical  test  shows 
that  only  for  a  small  range  of  operating  speeds,  about  a  quarter  of  the  maximum 
operating  speed,  the  static  simulation  can  provide  a  reasonable  estimate  for  the  maximum 
force  in  the  suspension  system.  Additionally,  this  figure  shows  that  this  conclusion  holds 

for  any  reasonable  value  of  A  selected  by  the  user  since  Fdmax(T)  increases  rapidly  at 
higher  velocities.  Typical  operating  speed  range  for  the  UGV  is  between  0  and  6 
miles/hour. 


5.6  Statistical  Assessment  of  Safe  Operating  Speed  Using  Dynamic  Simulation 

In  the  previous  section,  it  was  shown  that  the  static  simulation  fails  to  provide 
accurate  results.  Unlike  the  torque  saturation  scenario,  the  mean  of  dynamic  forces  in  the 
bumpy  road  setting  quickly  diverged  from  their  static  counterparts  within  typical 
operating  speeds.  Consequently,  we  use  the  dynamic  simulation  to  develop  boundaries 
of  safe  operating  speed. 


UGV  Speed  as  a  percentage  of  Maximum  Velocity  (%) 

Figure  13:  Statistical  test  for  comparison  of  dynamic  and  static  simulations  for  COr  =4,  A  = 
Fsmax  x  5%  ,  n=  10,  a  =  0.99. 
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As  mentioned  earlier,  three  failure  modes  associated  with  this  scenario  are 
intermittent  loss  of  contact  between  the  wheels  and  the  road,  excessive  mechanical 
shocks,  and  finally  flip  over.  The  intermittent  contact  is  an  early  indication  of  more 
severe  failures  such  as  mechanical  shocks.  Therefore,  this  event  can  provide  suitable 
insight  for  failure  prevention.  Flip  over,  on  the  other  hand,  occurs  at  higher  operating 
speeds,  while  severe  mechanical  shocks  mainly  take  place  at  velocities  close  to  the  onset 
of  flip  over.  Figure  14  shows  the  relation  among  the  three  events.  This  graph  also 
provides  a  general  guideline  for  how  the  UGV  should  be  operated  on  various  road 
conditions.  Since  intermittent  loss  of  contact  does  not  always  result  in  an  actual  failure, 
(but  rather  is  an  early  indication  of  other  failures),  to  develop  boundaries  of  safe 
operating  speeds,  we  will  focus  on  the  failure  mode  associated  with  excessive  mechanical 
shocks.  This  failure  is  directly  correlated  with  the  maximum  force  that  the  UGV 
suspension  experiences  on  a  road.  The  corresponding  hypothesis  test  for  failure  analysis 
is  as  follows: 


H° :  Vrvi 

Hl:Mr 


—  ^failure  threshold 


(V) 


>f< 


(20) 


failure  threshold 


In  which, 


failure  thre  shold 


is  the  failure  threshold  above  which  the  suspension  failure 


takes  place.  The  hypothesis  test  is  rejected  if: 


^d  (F)  ^failure  threshold 


SF 

4n 


>tt 


a,n-l 


(21) 


We  can  rewrite  inequality  (21)  to  obtain  a  rejection  boundary  for  the  hypothesis  test 
as  follows: 


77  max 


SF 

(F)  —  ^failure  threshold 


(22) 


Figure  15,  shows  the  Fdmax(F)  measurements  along  with  the  corresponding  rejection 
boundary  from  (22).  In  this  figure,  it  is  assumed  that  the  failure  occurs  when  the  robot 
experiences  forces  in  excess  of  400V,  i.e.,  Vfailurethreshold ,  which  is  more  than  3  times  as 

large  as  Fdmax(F)  at  low  velocities.  Given  the  current  failure  limit,  the  allowable 
operating  speed  range  on  this  bumpy  road  is  about  62%  of  the  maximum  operating 
velocity.  As  a  result,  the  failure  range  will  include  any  velocities  above  this  speed 
threshold.  This  failure  range,  which  can  result  in  costly  damages  to  the  UGV 
components,  can  then  be  excluded  from  the  acceptance  testing  immediately.  A  similar 
type  of  analysis  can  be  easily  applied  to  roads  with  different  surface  roughness,  bump 
size  and  road  profiles. 

To  summarize,  we  showed  that  the  dynamic  simulation  results  provide  general 
guidelines  to  how  the  UGV  should  be  operated  on  various  rough  bumpy  roads,  and  can 
significantly  help  with  failure  prevention  and  design  of  more  effective  physical 
acceptance  testing  scenarios. 
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Interrelation  Among  Different  Failure 
Modes 
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Figure  14:  Loss  of  contact  is  an  early  indication  of  suspension  system  and  flip  over  failures 
based  on  dynamic  simulation. 
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UGV  Speed  as  a  percentage  of  Maximum  Velocity  (%) 

Figure  15:  boundaries  of  safe  operation  for  suspension  breakdown  and  flip  over  prevention  on  a 

typical  bumpy  road  with  mild  roughness  for  COr  =  4  based  on  dynamic  simulation. 

6  CONCLUSIONS  AND  FUTURE  WORK 

In  this  paper,  we  introduced  a  simulation-based  acceptance  testing  methodology  that 
can  be  used  as  a  complementary  tool  for  physical  testing  under  a  number  of  typical 
operational  failures,  which  include  joint  torque  saturation,  rollover,  flip  over,  and 
suspension  system  breakdown.  To  capture  operational  variations  and  environmental 
uncertainties,  a  statistical  hypothesis  test  is  proposed  to  quantitatively  test  whether  the 
difference  between  dynamic  simulations  and  static  simulations  significantly  affect  the 
decision  of  UGV  operational  states.  The  results  of  statistical  hypothesis  testing  show  that 
in  the  analysis  of  joint  torque  saturation  and  rollover  failures,  that  are  operated  under 
slow  operational  speeds,  the  decisions  on  the  failure  states  are  consistent  between  the 
static  and  dynamic  simulations.  Therefore,  static  simulations  can  be  used  as  a  good 
approximation  of  dynamic  simulations  in  these  situations.  In  contrast,  in  the  case  of 
analyzing  the  flip  over  and  suspension  failures,  dynamic  simulations  cannot  be 
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approximated  by  static  simulations.  Moreover,  we  further  show  how  to  develop  the 
boundaries  of  safe  operation  based  on  the  simulation  results.  These  results  can  help 
provide  a  guideline  in  the  design  of  efficient  physical  testing  scenarios,  which  not  only 
help  avoid  some  unnecessary  testing  scenarios,  but  also  can  avoid  some  scenarios  leading 
to  severe  damages  to  UGVs  during  the  physical  tests. 

MSC  ADAMS,  which  was  used  to  compare  the  static  and  dynamic  simulation  results 
in  the  paper,  is  a  widely  used  dynamic  simulation  tool  in  industry.  Recently,  Pepper, 
Balakirsky,  &  Scrapper  (2007)  performed  the  physical  validation  tests  for  the  Talon 
Robot’s  simulation  models.  They  investigated  the  similarities  and  differences  of  the 
simulation  results  from  the  physical  tests  for  improving  the  simulation  models  and  their 
parameters  under  the  specific  scenarios,  such  as  passing  through  steps  and  stairs,  or 
directed  perception  and  grasping  dexterity  tests.  However,  there  are  no  physical  tests 
conducted  for  the  torque  saturation  and  flip-over  failures  that  are  presented  in  this  paper. 
So,  future  work  will  be  needed  to  experimentally  validate  those  dynamic  simulation 
models  using  the  physical  facilities  such  as  those  available  at  NIST  (Messina  &  Jacoff, 
2006),  and  the  U.S.  Army  Aberdeen  Proving  Ground  (Tricomo,  2009). 
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